Linguistic Categories as Basins of Curvature

Linguistic grammars do a good job of elucidating the lexical categories and phrasal units of the languages they model. Certain recurrent Connectionist networks can model similar data but it is not easy to discern the abstract structures of the resulting representations. Hierarchical clustering is helpful in this regard, but it often produces implausible clusters and there seems to be no principled way of deciding which clusters are relevant. A more promising approach is to examine the curvature of the hidden space image in the output space. Because the relative frequency distributions of lexical items tend to be highly categorical in nature, with probabilistic variation in only a few dimensions, they generally lie on the linear surfaces of the output space. Consequently, categories are associated with regions of low curvature. I show that categoryextraction based on this principle can produce a more revealing analysis than hierarchical clustering in certain cases.